Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation

Existing approaches to automatic VerbNet-style verb classification are heavily dependent on feature engineering and therefore limited to languages with mature NLP pipelines. In this work, we propose a novel cross-lingual transfer method for inducing VerbNets for multiple languages. To the best of our knowledge, this is the first study which demonstrates how the architectures for learning word embeddings can be applied to this challenging syntactic-semantic task. Our method uses cross-lingual translation pairs to tie each of the six target languages into a bilingual vector space with English, jointly specialising the representations to encode the relational information from English VerbNet. A standard clustering algorithm is then run on top of the VerbNet-specialised representations, using vector dimensions as features for learning verb classes. Our results show that the proposed cross-lingual transfer approach sets new state-of-the-art verb classification performance across all six target languages explored in this work.Comment: EMNLP 2017 (long paper

Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigation

Traditional NLP has long held (supervised) syntactic parsing necessary for successful higher-level language understanding. The recent advent of end-to-end neural language learning, self-supervised via language modeling (LM), and its success on a wide range of language understanding tasks, however, questions this belief. In this work, we empirically investigate the usefulness of supervised parsing for semantic language understanding in the context of LM-pretrained transformer networks. Relying on the established fine-tuning paradigm, we first couple a pretrained transformer with a biaffine parsing head, aiming to infuse explicit syntactic knowledge from Universal Dependencies (UD) treebanks into the transformer. We then fine-tune the model for language understanding (LU) tasks and measure the effect of the intermediate parsing training (IPT) on downstream LU performance. Results from both monolingual English and zero-shot language transfer experiments (with intermediate target-language parsing) show that explicit formalized syntax, injected into transformers through intermediate supervised parsing, has very limited and inconsistent effect on downstream LU performance. Our results, coupled with our analysis of transformers' representation spaces before and after intermediate parsing, make a significant step towards providing answers to an essential question: how (un)availing is supervised parsing for high-level semantic language understanding in the era of large neural models

Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization

Semantic specialization is the process of fine-tuning pre-trained distributional word vectors using external lexical knowledge (e.g., WordNet) to accentuate a particular semantic relation in the specialized vector space. While post-processing specialization methods are applicable to arbitrary distributional vectors, they are limited to updating only the vectors of words occurring in external lexicons (i.e., seen words), leaving the vectors of all other words unchanged. We propose a novel approach to specializing the full distributional vocabulary. Our adversarial post-specialization method propagates the external lexical knowledge to the full distributional space. We exploit words seen in the resources as training examples for learning a global specialization function. This function is learned by combining a standard L2-distance loss with an adversarial loss: the adversarial component produces more realistic output vectors. We show the effectiveness and robustness of the proposed method across three languages and on three tasks: word similarity, dialog state tracking, and lexical simplification. We report consistent improvements over distributional word vectors and vectors specialized by other state-of-the-art specialization frameworks. Finally, we also propose a cross-lingual transfer method for zero-shot specialization which successfully specializes a full target distributional space without any lexical knowledge in the target language and without any bilingual data.Comment: Accepted at EMNLP 201

Structograms – A New Approach to Documenting the Quality Management System

Djelotvornost suvremenih sustava upravljanja kvalitetom temeljenih na normi ISO 9001:2008 znatno ovisi o jednostavnosti i jasnoći njihove dokumentacije, posebice dokumentiranih postupaka kvalitete (procedura). U radu se predlaže uporaba struktograma (u literaturi često zvanih Nassi-Shneidermanovi dijagrami) za prikaz slijeda radnja u postupcima kvalitete. Uspoređuju se dva načina prikaza, struktogramima i uobičajenim dijagramima toka. Prikazani su, objašnjeni i uspoređeni njihovi osnovni elementi i načela, uz prikaz prednosti i nedostataka. Podrobnija usporedba provedena je na stvarnom primjeru postupka kvalitete za specifikaciju materijala u brodograđevnoj proizvodnji. Zaključuje se da struktogrami uistinu imaju mnoge prednosti u usporedbi s dijagramima toka, te da treba poticati njihovu uporabu.Effectiveness and efficiency of modern quality management systems, based upon ISO 9001:2008 standard significantly depend upon simplicity and clarity of their documentation, especially documented quality procedures. This paper proposes implementation of structograms (or more frequently called Nassi-Shneiderman diagrams) for presentation of sequence of activities in quality procedures. Two ways of presentation are compared, i.e. structograms vs. conventional flowcharts. Their fundamental elements and principals are shown, explained and compared, with the presentation of advantages and drawbacks. Detailed comparison was made on real life example of a documented procedure for material specification in shipbuilding. It has been concluded that structograms really do have many benefits compared to flow charts and that their use is to be preferred

UTJECAJ FIRST-LINE MANAGEMENTA NA MOTIVACIJU ZAPOSLENIKA U PODUZEĆU „GALEB D.D.“ : Završni rad

Predmet ovog istraţivanja je utjecaj first-line, operativnog managementa na opću razinu motivacije zaposlenika na primjeru poduzeća Galeb d.d. Objašnjeni su pojmovi i vaţnost motivacije te su navedene teorije motivacije i motivacijske strategije. Definirane su managerske vještine i odgovornost, kao i aktivnosti djelovanja managera na odreĊenim podruĉjima. Naglasak je stavljen na dinamiĉnost i kompleksnost ljudskog resursa te da za uspješno voĊenje management mora prepoznati vaţnost kontinuiranog odrţavanja ţeljene razine motivacije zaposlenika. Anketnim upitnikom kao primarnim izvorom informacija omogućen je detaljan uvid u stavove zaposlenika. Ovisno o kvaliteti dobivenih rezultata management moţe analizirati informacije i kreirati strategije za odrţavanje pozitivnih faktora i eliminiranje uzroĉnika smanjenja motivacije. Specifiĉnost ovog istraţivanja leţi u tome što prikazuje udjele nezadovoljstva radnika po ciljanim kategorijama i daje uvid u veliĉinu utjecaja odreĊenih problema na motivaciju zaposlenika. Većina ovih problema moţe se eliminirati bez materijalnih troškova, iskljuĉivo kombinacijom uroĊenih i nauĉenih vještina managementa. Kvalitetan management ništa ne prepušta sluĉaju, još ako ima mogućnost provoditi jeftine i efikasne poteze za unaprjeĊenja radne okoline, onda tu mogućnost svakako treba iskoristiti na što ovaj rad skreće paţnju.The object of this research is to inspect the influence of first-line management on the general level of employee motivation in the case of Galeb d.d. It explains the importance of motivation and states various theories and motivation strategies. It defins the skills and knowledge of a propper manager, as well as possible moves he/she can make in order to deal with a certain situation. Throughout the paper it has been made clear that the nature of human resources is very dynamic and in order to functionally operate at a desired level, management must realize the importance of the issue and then try to anticipate anything that could effect the motivation levels of the employees. Using the survey as a primary source of information, it's possible to get a full understanding of the employees opinions, values and senses of appreciation. All mentioned and many more factors influence the motivation levels on a daily basis. Based on the quality of gathered informations, management has the tools necessary to eliminate the risks and to maximize the benefits. What's so special about this research is that it provides numerical and graphical shares of employee's (di)satisfaction thus showing the volume of impact the specific problem has on the motivation levels. Most of the problems detected are relatively easy to solve. Given the situation, management should never hesitate to make action if the possible outcome far exceeds the cost or difficulty of the issue, as it is the case in this research

Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval

State-of-the-art neural (re)rankers are notoriously data hungry which - given the lack of large-scale training data in languages other than English - makes them rarely used in multilingual and cross-lingual retrieval settings. Current approaches therefore typically transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders: they fine-tune all the parameters of a pretrained massively multilingual Transformer (MMT, e.g., multilingual BERT) on English relevance judgments and then deploy it in the target language. In this work, we show that two parameter-efficient approaches to cross-lingual transfer, namely Sparse Fine-Tuning Masks (SFTMs) and Adapters, allow for a more lightweight and more effective zero-shot transfer to multilingual and cross-lingual retrieval tasks. We first train language adapters (or SFTMs) via Masked Language Modelling and then train retrieval (i.e., reranking) adapters (SFTMs) on top while keeping all other parameters fixed. At inference, this modular design allows us to compose the ranker by applying the task adapter (or SFTM) trained with source language data together with the language adapter (or SFTM) of a target language. Besides improved transfer performance, these two approaches offer faster ranker training, with only a fraction of parameters being updated compared to full MMT fine-tuning. We benchmark our models on the CLEF-2003 benchmark, showing that our parameter-efficient methods outperform standard zero-shot transfer with full MMT fine-tuning, while enabling modularity and reducing training times. Further, we show on the example of Swahili and Somali that, for low(er)-resource languages, our parameter-efficient neural re-rankers can improve the ranking of the competitive machine translation-based ranker

Quantifying the Dialect Gap and its Correlates Across Languages

Historically, researchers and consumers have noticed a decrease in quality when applying NLP tools to minority variants of languages (i.e. Puerto Rican Spanish or Swiss German), but studies exploring this have been limited to a select few languages. Additionally, past studies have mainly been conducted in a monolingual context, so cross-linguistic trends have not been identified and tied to external factors. In this work, we conduct a comprehensive evaluation of the most influential, state-of-the-art large language models (LLMs) across two high-use applications, machine translation and automatic speech recognition, to assess their functionality on the regional dialects of several high- and low-resource languages. Additionally, we analyze how the regional dialect gap is correlated with economic, social, and linguistic factors. The impact of training data, including related factors like dataset size and its construction procedure, is shown to be significant but not consistent across models or languages, meaning a one-size-fits-all approach cannot be taken in solving the dialect gap. This work will lay the foundation for furthering the field of dialectal NLP by laying out evident disparities and identifying possible pathways for addressing them through mindful data collection.Comment: Accepted to EMNLP Findings 202

CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models

While many languages possess processes of joining two or more words to create compound words, previous studies have been typically limited only to languages with excessively productive compound formation (e.g., German, Dutch) and there is no public dataset containing compound and non-compound words across a large number of languages. In this work, we systematically study decompounding, the task of splitting compound words into their constituents, at a wide scale. We first address the data gap by introducing a dataset of 255k compound and non-compound words across 56 diverse languages obtained from Wiktionary. We then use this dataset to evaluate an array of Large Language Models (LLMs) on the decompounding task. We find that LLMs perform poorly, especially on words which are tokenized unfavorably by subword tokenization. We thus introduce a novel methodology to train dedicated models for decompounding. The proposed two-stage procedure relies on a fully self-supervised objective in the first stage, while the second, supervised learning stage optionally fine-tunes the model on the annotated Wiktionary data. Our self-supervised models outperform the prior best unsupervised decompounding models by 13.9% accuracy on average. Our fine-tuned models outperform all prior (language-specific) decompounding tools. Furthermore, we use our models to leverage decompounding during the creation of a subword tokenizer, which we refer to as CompoundPiece. CompoundPiece tokenizes compound words more favorably on average, leading to improved performance on decompounding over an otherwise equivalent model using SentencePiece tokenization.Comment: EMNLP 202

Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentation

Many NLP pipelines split text into sentences as one of the crucial preprocessing steps. Prior sentence segmentation tools either rely on punctuation or require a considerable amount of sentence-segmented training data: both central assumptions might fail when porting sentence segmenters to diverse languages on a massive scale. In this work, we thus introduce a multilingual punctuation-agnostic sentence segmentation method, currently covering 85 languages, trained in a self-supervised fashion on unsegmented text, by making use of newline characters which implicitly perform segmentation into paragraphs. We further propose an approach that adapts our method to the segmentation in a given corpus by using only a small number (64-256) of sentence-segmented examples. The main results indicate that our method outperforms all the prior best sentence-segmentation tools by an average of 6.1% F1 points. Furthermore, we demonstrate that proper sentence segmentation has a point: the use of a (powerful) sentence segmenter makes a considerable difference for a downstream application such as machine translation (MT). By using our method to match sentence segmentation to the segmentation used during training of MT models, we achieve an average improvement of 2.3 BLEU points over the best prior segmentation tool, as well as massive gains over a trivial segmenter that splits text into equally sized blocks.Comment: ACL 202